Skip to main content

Custom Crawl Services

Internet Archive

National library harvesting.
SHOW DETAILS
Title
Date Archived
Creator
45.1M
Crawls performed by Internet Archive on behalf of the National Library of Australia. This data is currently not publicly accessible.
National Library of Spain
6,695
ITEMS
35.5M
VIEWS
35.5M
Data collected by Internet Archive on behalf of the National Library of Spain. This data is currently not publicly accessible.
34.7M
Crawls of the french domain space performed by Internet Archive on behalf of Bibliotheque Nationale de France. This data is currently not publicly accessible.
Elections Web
1,604
ITEMS
17.4M
VIEWS
17.4M
This collection contains collaborative Election crawls performed by IA.
Topics: elections, web
Election Crawl 2012
1,603
ITEMS
17.4M
VIEWS
17.4M
This crawl was performed in Summer & Fall of 2012 to archive the US Federal Elections.
Topics: US, federal, elections, web, 2012
bnf_2008
714
ITEMS
16.7M
VIEWS
16.7M
this data is currently not publicly accessible.
16M
National Library of Austrailia crawl. This data is currently not publicly accessible.
11.9M
National Archives and Records Administration crawl performed by Internet Archive. This data is currently not publicly accessible.
Olympics Web
2,008
ITEMS
10.6M
VIEWS
10.6M
This collection includes all collaborative Olympic crawls performed by IA for the IIPC.
Topics: olympics, IIPC, web
NLA 2013 Domain crawl
2,796
ITEMS
8M
VIEWS
8M
This crawl of the .au domain was performed on behalf of the National Library of Australia in Spring of 2013.
Topics: nla, web, 2013
NLS_2011
1,500
ITEMS
7.9M
VIEWS
7.9M
These crawls of the .es domain were performed in 2011 on behalf of the National Library of Spain (BNE).
Topics: bne, spain, web, 2011
nls_2010
971
ITEMS
7.8M
VIEWS
7.8M
this data is currently not publicly accessible.
Olympics Crawl 2012
700
ITEMS
7.8M
VIEWS
7.8M
These crawls were performed by IA on behalf of the IIPC in Summer 2012 during and prior to the 2012 Summer Olympics held in London, UK.
Topics: London, olympics, web, 2012, IIPC
bnf_2007
321
ITEMS
7.8M
VIEWS
7.8M
this data is currently not publicly accessible.
nls_2009
872
ITEMS
6.9M
VIEWS
6.9M
this data is currently not publicly accessible.
nla_2008
628
ITEMS
6.5M
VIEWS
6.5M
this data is currently not publicly accessible.
6.2M
This crawl of online resources of the 112th US Congress was performed in Fall of 2012 and early winter of 2013 on behalf of NARA.
Topics: nara, 112th, web
6M
Crawls performed by Internet Archive on behalf of the National Library of New Zealand. This data is currently not publicly accessible.
nla_2007
369
ITEMS
5.6M
VIEWS
5.6M
this data is currently not publicly accessible.
nla_2009
566
ITEMS
5.5M
VIEWS
5.5M
this data is currently not publicly accessible.
Topics: bne, spain, web, 2013
NLA_2014
2,150
ITEMS
5.4M
VIEWS
5.4M
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2014.
Topics: nla, web, 2014
bnf_2005
265
ITEMS
5.3M
VIEWS
5.3M
this data is currently not publicly accessible.
nla_2006
384
ITEMS
5.3M
VIEWS
5.3M
this data is currently not publicly accessible.
National Library of Israel
2,058
ITEMS
5.2M
VIEWS
by dominic@archive.org
5.2M
Data collected by Internet Archive on behalf of the National Library of Israel.  This data is currently not publicly accessible.
Topic: nlil
nla_2005
175
ITEMS
5.1M
VIEWS
5.1M
this data is currently not publicly accessible.
bnf_2006
322
ITEMS
4.6M
VIEWS
4.6M
this data is currently not publicly accessible.
NLIL_2013
1,185
ITEMS
4.4M
VIEWS
by dominic@archive.org
4.4M
This crawl of the .il domain was performed in 2013 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2013
NLS_2012
769
ITEMS
4.2M
VIEWS
4.2M
This crawl of the .es domain was performed in 2012 on behalf of the National Library of Spain (BNE).
Topics: bne, spain, web, 2012
National Library of Sweden
309
ITEMS
4.1M
VIEWS
4.1M
Data collected by Internet Archive on behalf of the National Library of Sweden. This data is currently not publicly accessible.
nl_sweden_2010
308
ITEMS
4.1M
VIEWS
4.1M
this data is currently not publicly accessible.
nlnzweb2013
912
ITEMS
3.8M
VIEWS
3.8M
This collection includes content harvested from the Web on behalf of the National Library & Archives New Zealand in February 2013.
Topics: web, domain
3.2M
This crawl of online resources of the 111th Congress of the United States was performed in Fall of 2010 and Winter of 2011 on behalf of NARA.
Topics: nara, 111th, congress, web
3M
Data collected by Internet Archive on behalf of Biblioteca Nazionale Centrale di Firenze. This data is currently not publicly accessible.
2.8M
Crawls performed by Internet Archive on behalf of the National Library of Ireland. This data is currently not publicly accessible.
nli_2007
62
ITEMS
2.4M
VIEWS
2.4M
this data is currently not publicly accessible.
Fed Site Closure Crawls
1,840
ITEMS
2.1M
VIEWS
2.1M
These are crawls performed on US Federal Government Web sites prior to their removal or merge with other resources.
Topics: federal, web, closures
Fed Site Closures 2011
1,839
ITEMS
2.1M
VIEWS
2.1M
This crawl was performed in Fall of 2011 to archive Federal government web sites that were either slated for removal or for merger with other online resources.
Topics: federal, web, 2011
Olympics Crawl 2014
1,318
ITEMS
2.1M
VIEWS
2.1M
These crawls were performed by IA on behalf of the IIPC in Winter 2014 during and prior to the 2014 Winter Olympics and Paralympic Games held in Sochi, Russia.
Topics: olympics 2014, web, sport, olympic games
NLA_2010
178
ITEMS
2M
VIEWS
2M
This crawl was a domain scale harvest of .au performed for the National Library of Australia in 2010.
Topics: nla, web, 2010
NLS_elec2011
279
ITEMS
1.9M
VIEWS
1.9M
This crawl was performed on behalf of the National Library of Spain (BNE) in Fall of 2011 to archive the National elections in Spain.
Topics: elections, web, 2011, spain, bne
nlnz_2010
167
ITEMS
1.6M
VIEWS
1.6M
this data is currently not publicly accessible.
NLS_humanidades
296
ITEMS
1.2M
VIEWS
1.2M
This crawl was performed in 2011 and 2012 on behalf of the National Library of Spain (BNE) to archive digital humanities web sites and online resources in Spain.
Topics: bne, spain, web, humanities, humanidades, 2011, 2012
NLAgov_2010
615
ITEMS
1.2M
VIEWS
1.2M
This crawl was performed on the .gov.au domain in 2010 on behalf of the National Library of Australia.
Topics: nla, gov.web, 2010
UNT Web
35
ITEMS
1.1M
VIEWS
1.1M
This collection contains all collaborative crawl data contributed by University of North Texas (UNT).
Topics: UNT, web, texas, eot
NLIL_2014
971
ITEMS
950,683
VIEWS
by dominic@archive.org
950,683
This crawl of the .il domain was performed in 2014 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2014
Olympics Crawl 2010
21
ITEMS
792,496
VIEWS
792,496
These crawls were performed by IA on behalf of the IIPC in Winter 2010 during and prior to the 2010 Winter Olympics held in Vancouver, BC, Canada.
Topics: winter, olympics, 2010, IIPC, web
by Internet Archive
720,860
This collection includes all resources harvested from the online presence of the Legislative branch of the US Federal government as part of the NARA 112th Congressional Web Harvest Test Crawl. The crawl was performed from October 16th through November 5th 2012.
Topics: NARA, 112th, Congress
691,057
Data collected by Internet Archive on behalf of the Fundacao para a Computacao Cientifica Nacional of Portugal. This data is currently not publicly accessible.
nlnz_2008
97
ITEMS
677,445
VIEWS
677,445
this data is currently not publicly accessible.
NDIIPP Youtube Crawl
90
ITEMS
635,410
VIEWS
635,410
Youtube crawl performed by Internet Archive on behalf of the National Digital Internet Infrastructure Preservation Program. This data is currently not publicly accessible.
NLA_2015
3,087
ITEMS
480,342
VIEWS
480,342
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2015.
Topics: nla, web, 2015
Nara 110th Congressional Crawl
106
ITEMS
416,296
VIEWS
416,296
The end of term harvest of the 110th Congress of the United States was performed on behalf of NARA in Fall of 2008 and early winter of 2009.
Topics: nara, 110th, congress, web
nli_2008
45
ITEMS
398,058
VIEWS
398,058
this data is currently not publicly accessible.
nlnzweb2015
1,071
ITEMS
375,757
VIEWS
375,757
This collection includes content harvested from the Web on behalf of the National Library & Archives New Zealand in January 2015.
Topics: new zealand, web, domain
Data crawled by University of North Texas on behalf of University of North Texas from Tue Sep 16 10:16:10 PDT 2008 to Tue Sep 16 11:32:37 PDT 2008
Topic: crawldata
NLS_2011
by Internet Archive
306,569
0
0
Internet Archive crawldata uploaded by selenium-102.us.archive.org:NLS-CRAWL-002 from Sat Jul 2 06:39:17 PDT 2011 to Thu Dec 13 04:01:50 PST 2012.
Topic: crawldata
Data crawled by Fundacao para a Computacao Cientifica Nacional on behalf of Internet Archive from Mon Aug 30 00:00:00 PDT 2010 to Mon Aug 30 00:00:00 PDT 2010
Topic: crawldata
bnf_2004
22
ITEMS
266,093
VIEWS
266,093
this data is currently not publicly accessible.
Data crawled by National Library of Australia on behalf of Internet Archive from Thu Jun 16 05:39:37 PDT 2005 to Thu Jun 16 08:45:17 PDT 2005
Topic: crawldata
Data crawled by National Library of Australia on behalf of Internet Archive from Thu Jun 23 20:24:49 PDT 2005 to Fri Jun 24 03:05:29 PDT 2005
Topic: crawldata
Data crawled by National Library of Australia on behalf of Internet Archive from Sat Oct 17 18:51:39 PDT 2009 to Sat Oct 17 20:59:29 PDT 2009
Topic: crawldata
204,821
This crawl of online resources of the 113th US Congress was performed on behalf of NARA.
Data crawled by National Library of Australia on behalf of Internet Archive from Tue Aug 29 10:54:17 PDT 2006 to Tue Aug 29 12:26:12 PDT 2006
Topic: crawldata
Data crawled by University of North Texas on behalf of University of North Texas from Sat Sep 20 12:47:10 PDT 2008 to Sat Sep 20 14:29:46 PDT 2008
Topic: crawldata
Data crawled by National Library of Australia on behalf of Internet Archive from Thu Jun 23 13:21:19 PDT 2005 to Thu Jun 23 20:24:49 PDT 2005
Topic: crawldata
Data crawled by National Library of Ireland on behalf of Internet Archive from Sat Nov 17 07:03:25 PDT 2007 to Sat Nov 17 09:46:18 PDT 2007
Topic: crawldata
Internet Archive crawldata uploaded by selenium-102.us.archive.org:NARA-111TH from Fri Dec 17 07:12:05 PST 2010 to Mon Nov 12 09:10:30 PST 2012.
Topic: crawldata
nlnzweb2013
181,915
0
0
Internet Archive crawldata from National Library & Archives New Zealand, captured by wbgrp-crawl007.us.archive.org:NLNZ-NZ-CRAWL-003 from Thu Feb 28 06:07:52 PST 2013 to Mon Mar 4 17:03:17 PST 2013.
Topic: crawldata
Data crawled by Bibliothque Nationale de France on behalf of Internet Archive from Fri Apr 04 22:43:24 PDT 2008 to Sat Apr 05 01:50:32 PDT 2008
Topic: crawldata
Data crawled by National Library of Ireland on behalf of Internet Archive from Sat Nov 17 22:17:22 PDT 2007 to Sun Nov 18 03:10:14 PDT 2007
Topic: crawldata
Data crawled by National Library of Australia on behalf of Internet Archive from Thu Jun 23 06:28:22 PDT 2005 to Thu Jun 23 13:18:26 PDT 2005
Topic: crawldata
Data crawled by National Library of Australia on behalf of Internet Archive from Wed Oct 14 15:56:51 PDT 2009 to Wed Oct 14 16:57:20 PDT 2009
Topic: crawldata
nls_2010
by Internet Archive
159,587
0
0
Internet Archive crawldata uploaded by selenium-102.us.archive.org:NLS-CRAWL-001C from Wed Oct 20 21:57:16 PDT 2010 to Sat Jan 12 19:05:23 PST 2013.
Topic: crawldata
Data crawled by Bibliothque Nationale de France on behalf of Internet Archive from Wed Oct 17 09:46:53 PDT 2007 to Wed Oct 17 11:03:57 PDT 2007
Topic: crawldata